A wide variety of sophisticated software debugging and analysis tools have been developed during the past forty years to facilitate design, testing, and analysis of software programs. Generally, a software program is developed through an iterative process, each stage of which consists of program design or modification followed by testing and analysis of the program. A commonly encountered technique for analyzing software programs is to embed instrumentation within the program for collecting various types of data while the program is running. Data can be collected and processed in various ways to provide a basis for analyzing the program. Often, embedded instrumentation provides the only practical approach to understanding the real-time behavior of complex programs and automated systems. For example, a running web site may field requests for hypertext markup language (“HTML”) documents at a very high rate, and may concurrently store information provided by remote users in a database, conduct commercial transactions with remote users, and perform other such tasks. Although the web site programs may be well tested and understood, the real-time, dynamic environment in which they operate may not be. By instrumenting the web site, web site developers and managers can determine temporal patterns of HTML-page-request reception, statistics on requests received for the HTML pages served, statistics on transactions carried out, and other such information. The collected data can be used to configure the web site to more efficiently handle requests and transactions, to discover program errors, to undertake directed marketing campaigns to advertise underutilized features of the website, and to exploit discovered relationships within purchasing behaviors of different categories of website users for targeted cross-marketing campaigns.
The application response measurement (“ARM”) standard was developed as a measurement standard for instrumenting software programs. The ARM standard specifies the semantics of six ARM routines that allow a program developer to embed instrumentation, in the form of transactions, within a program. These transactions can be used to record data and transmit the recorded data to a data processing routine or system for analysis. The ARM standard is relatively simple and easy to use. It allows programmers to construct transaction types suitable for their instrumentation needs, and is therefore adaptable to different measurement tasks, and is widely available.
As a result of its simplicity, the ARM standard suffers from notable deficiencies. First, although many different types of instrumentation might be desirable for inclusion within a program, the ARM standard provides only a transaction measurement type. Second, the ARM standard does not specify many details with regard to encoding and transportation of data, collected via embedded instrumentation from a program, to a data processing and analysis program or system. Third, the ARM standard lacks convenient facilities for locally filtering and partitioning of collected data in order to decrease the data transmission overhead between the running program and a data processing or data analysis program or system and in order to decrease data collection and data processing overhead within the instrumented program and analysis system, respectively. Thus, although the ARM standard has proved to be a valuable and widely used improvement over the various ad hoc instrumentation techniques that preceded the ARM standard, software program developers have recognized the need for a standard, but more completely defined and more flexible software program instrumentation method and system for embedding instrumentation within software programs, collecting data via the embedded instrumentation, packaging the data, and transmitting the packaged data to a data processing and/or data analysis routine.